Reducing overconfidence in forecasting with repeated judgement elicitation
نویسندگان
چکیده
Overconfidence is the tendency for people to underestimate the true range of uncertainty regarding unknown or future values. It results in observed outcomes falling outside people’s estimated ranges more often than their stated confidence would suggest. Previous research has, however, demonstrated various ways of reducing this bias and the More-Or-Less-Elicitation (MOLE) tool has been designed to take these into account while leading people through an elicitation. Previous research showed MOLE’s benefit on a visual estimation task but real world elicitation is more likely to involve forecasting future values. The current study compared forecast ranges, for 7 and 28 day windows, elicited via the MOLE and direct estimation. A significant reduction in overconfidence (the mismatch between stated confidence and the proportion of ranges containing the true value) was observed – from more than 25% to only 7%. We conclude that the MOLE is a useful tool for assisting forecasting.
منابع مشابه
More-Or-Less Elicitation (MOLE): Testing A Heuristic Elicitation Method
Elicitation of people’s knowledge is a central methodological challenge for psychology, with important impacts in many technical disciplines and industrial settings. The need to convert an expert’s beliefs into a useable format is of particular importance when judgments and decisions are made under uncertainty. Simply asking a person for their best estimate or to estimate a range is subject to ...
متن کاملReducing overconfidence in the interval judgments of experts.
Elicitation of expert opinion is important for risk analysis when only limited data are available. Expert opinion is often elicited in the form of subjective confidence intervals; however, these are prone to substantial overconfidence. We investigated the influence of elicitation question format, in particular the number of steps in the elicitation procedure. In a 3-point elicitation procedure,...
متن کاملFinancial professionals ’ overconfidence : Is it experience , job , or attitude ?
This paper examines financial professionals’ overconfidence in their forecasting performance. We are the first to compare individual financial professionals’ self-ratings with their true forecasting performance. Data spans several years at monthly frequency. The forecasters in our sample do not provide feasible self-ratings compared to their true performance but show overconfidence on average. ...
متن کاملUnpacking the Future: A Nudge Toward Wider Subjective Confidence Intervals
Subjective probabilistic judgments are inevitable in many real life domains. A common way to obtain such judgments is to assess fractiles or confidence intervals. However, such judgments tend to be systematically overconfident. For example, 90% confidence intervals for future uncertain quantities (e.g., future stock prices) are likely to capture only 50-60% of the actual realizations. Furthermo...
متن کامل